On the Calibration of Aggregated Conformal Predictors
نویسندگان
چکیده
Conformal prediction is a learning framework that produces models that associate with each of their predictions a measure of statistically valid confidence. These models are typically constructed on top of traditional machine learning algorithms. An important result of conformal prediction theory is that the models produced are provably valid under relatively weak assumptions—in particular, their validity is independent of the specific underlying learning algorithm on which they are based. Since validity is automatic, much research on conformal predictors has been focused on improving their informational and computational efficiency. As part of the efforts in constructing efficient conformal predictors, aggregated conformal predictors were developed, drawing inspiration from the field of classification and regression ensembles. Unlike early definitions of conformal prediction procedures, the validity of aggregated conformal predictors is not fully understood—while it has been shown that they might attain empirical exact validity under certain circumstances, their theoretical validity is conditional on additional assumptions that require further clarification. In this paper, we show why validity is not automatic for aggregated conformal predictors, and provide a revised definition of aggregated conformal predictors that gains approximate validity conditional on properties of the underlying learning algorithm.
منابع مشابه
Comparing Performance of Different Inductive and Transductive Conformal Predictors Relevant to Drug Discovery
We present an evaluation of the impact of transductive, inductive, aggregated and cross inductive mondrian conformal prediction on the validity and efficiency of predictions. The aim of the study is to give guidance to which methods perform best where there is limited data. The evaluation has been made on a large public dataset of Ames mutagenicity data, relevant for drug discovery, a spam data...
متن کاملCombination of Conformal Predictors for Classification
The paper presents some possible approaches to the combination of Conformal Predictors in the binary classification case. A first class of methods is based on p-value combination techniques that have been proposed in the context of Statistical Hypothesis Testing; a second class is based on the calibration of p-values into Bayes factors. A few methods from these two classes are applied to a real...
متن کاملCP-RAkEL: Improving Random k-labelsets with Conformal Prediction for Multi-label Classification
Multi-label conformal prediction has attracted much attention in the conformal predictor (CP) community. In this article, we propose to combine CP with random k -labelsets (RAkEL) method, which is state-of-the-art multi-label classification method for large number of labels. In the framework of RAkEL, the original problem is reduced to a number of small-sized multi-label classification tasks by...
متن کاملOn Infinitesimal Conformal Transformations of the Tangent Bundles with the Generalized Metric
Let be an n-dimensional Riemannian manifold, and be its tangent bundle with the lift metric. Then every infinitesimal fiber-preserving conformal transformation induces an infinitesimal homothetic transformation on . Furthermore, the correspondence gives a homomorphism of the Lie algebra of infinitesimal fiber-preserving conformal transformations on onto the Lie algebra of infinitesimal ...
متن کامل